Skip to content

OnPair: decode straight into the builder's byte storage - #9100

Open
robert3005 wants to merge 1 commit into
rk/zstd-append-to-builderfrom
rk/onpair-decode-into-builder
Open

OnPair: decode straight into the builder's byte storage#9100
robert3005 wants to merge 1 commit into
rk/zstd-append-to-builderfrom
rk/onpair-decode-into-builder

Conversation

@robert3005

@robert3005 robert3005 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Decompress value straight into builders storage

@robert3005 robert3005 added the changelog/chore A trivial change label Jul 31, 2026
Comment thread encodings/onpair/src/canonical.rs Outdated
///
/// `try_decode_into` is generic over the dictionary view and only specializes its batched
/// copy loop once inlined into the caller, so leaving this wrapper out of line costs ~2.5x.
#[inline(always)]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no always

return result;
}

let Some(builder) = builder.as_any_mut().downcast_mut::<VarBinViewBuilder>() else {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be made better

@robert3005
robert3005 marked this pull request as ready for review July 31, 2026 14:07
Same shape as the FSST change: `append_to_builder` staged the decoded heap in a
temporary buffer before copying it in. `OnPairDecodePlan` hoists the code
window, the dictionary view, the per-row lengths and the total decoded size out
of `onpair_decode_bytes` so the decode can target the builder's spare capacity
via `append_decoded`, leaving one prefix sum over the lengths as the only work
beyond the bulk `try_decode_into`.

`decode_into` is `#[inline(always)]`: `try_decode_into` is generic over the
dictionary view and only specializes its batched copy loop once inlined into the
caller, so leaving the wrapper out of line costs ~2.5x.

Also extends the Arrow export benchmarks to cover OnPair alongside the other
string encodings, plus nullable variants and the two `append_to_builder`
benchmarks that reach an encoding's specialization directly — the Arrow export
cannot stand in for those, since `execute_until` canonicalizes a bare
FSST/OnPair/Zstd root before any builder sees it.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>
@robert3005
robert3005 force-pushed the rk/onpair-decode-into-builder branch from 9594b8b to 83b3f8a Compare July 31, 2026 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant